LED Panel Having Integrated Infrared Retroreflectors For Video Volume Production Environments

ABSTRACT

A LED panel wherein a plurality thereof are employed to construct a LED volume for filing simulated virtual environments wherein the LED panel of the present invention provides tracking of objects in the vicinity thereof such as but not limited to a camera wherein the tracking elements do not require post production removal. The LED panel includes a housing having a perimeter frame that has a transparent LED display mounted thereto. On the rear surface of the LED display or proximate thereto are a plurality of retroreflectors. The retroreflectors function to provide inside-out tracking of a camera disposed within the LED volume. The present invention further includes a plurality of lidar sensors and optical sensors mounted to the perimeter frame. The lidar sensors and optical sensors provide data for outside-in tracking of a camera within the LED volume. The retroreflectors can be provided in multiple alternate embodiments.

PRIORITY UNDER 35 U.S.C SECTION 119(E) & 37 C.F.R. SECTION 1.78

This nonprovisional application claims priority based upon the followingprior U.S. Provisional Pat. Application entitled: Integrated IRRetroreflector for LED Video Volume Production Environments, ApplicationNo.: 63/136,058 filed Jan. 11, 2021, in the name of Joseph Register,which is hereby incorporated by reference for all purposes.

FIELD OF THE INVENTION

The present invention relates generally to virtual environments for filmproduction, more specifically but not by way of limitation, a virtualfilm environment created with a plurality of LED panels wherein the LEDpanels of the present invention include a multitude of infraredretroreflectors embedded therein that are operable to provide threedimensional tracking of an object or camera.

BACKGROUND

The video industry commonly employs the utilization of LED walls tosimulate virtual environments wherein the LED walls produce aphotorealistic image on camera. These simulated virtual environmentsproduce an image that appears real on camera and are a cost effectivealternative to location based shooting. Utilization of these virtualenvironments allows a real time rendering of backgrounds and is asignificant improvement to conventional green screen technology whichrequires the background to be removed and rendered in post-production.The utilization of LED wall environments is referred to as virtualproduction or in camera visual effects. These LED walls are oftenconstructed in studios and referred to as LED volumes when built in theshape of a dome.

The LED volumes are composed of a multitude of LED wall panels and LEDceiling panels so as to simulate a full three hundred and sixty degreeenvironment of a virtual location. This technique offers advantages overtraditional green screen video techniques by providing improvedsimulated lighting and reflections that are nearly indistinguishablefrom live shots in these environments which results in more accuratefinal images. By way of example but not limitation, a vehicle filmed ona green screen has complex reflections of green that appear in themirrors, contours and windows and additionally has contamination ofgreen ambient light thereon. Whereas an LED volume portrays a real timebackground through the windows and mirrors resulting in the reflectionof the correct ambient light colors and visuals on the car in real time.This creates a level of realism that is far superior to the green screenscenes, which have to be manually created and manipulated inpost-production.

For the illusion to work, the main camera within the LED volume must betracked for position so that the environment can move appropriately tothe perspective of the camera. This provides a forced perspective tocreate the illusion that the shot was filmed from a particular point ofview that is a matching point of view of the objects such as but notlimited to props on the stage. For example, when filming a car on avirtual road, the angle of the film capture should change perspectivesin real time on the LED display from one lane to another when the camerais laterally moved from one lane to the other. For proper execution ofthe aforementioned, the camera’s location in three-dimensional spacemust be tracked wherein the positional data is updated in a computer sothat the proper perspective is displayed on the screen. This positionalfeedback loop that is created by using camera tracking is one of themost unique and important features to make virtual production appearreal.

Currently there are two optical tracking methodologies that are commonlyused for tracking the camera’s position. The first is referred to asoutside-in tracking and the second is referred to as inside-outtracking. In outside-in tracking, tracking devices are placed instationary locations surrounding a marker on the tracked camera todetermine its position in space by cross-referencing the position of themarker relative to the position of the tracking devices. These type ofsystems typically have multiple tracking devices that allow for a 360degree perspective of the tracking marker on the tracked camera, andthis overlap allows for accurate readings of the camera’s position. Thesecond existing methodology, inside-out tracking is the alternatemethodology in which the tracking emitter is placed directly on thecamera and looks outward to determine its location using markers withinthe room. These markers, often referred to as tracking stars, are signalreflectors used to triangulate the tracking emitters location inthree-dimensional space. Often times a star constellation of trackingstars is created so as to increase the amount of tracking points,wherein hundreds of randomized reflective tracking markers are placed onthe wall or ceiling to determine the camera’s position inthree-dimensional space.

The problem with the aforementioned existing technologies is inside theLED volume both of these methodologies rely on visible tracking elementsthat are often times visible in the final scene. The outside in cameratracking spoils the effect by appearing as a ring of tracking devices inthe final footage and the inside out methodology show up as spottyartifacts where the star constellation markers occlude the virtual imageon the LED screen. Ultimately, both of these methodologies requireeditors to remove these tracking devices or markers in post-production,which results in decreased workflow efficiency and increased cost. Thegoal of virtual production is to avoid the post-production process byrecording real time virtual environments in camera and avoid having toremove tracking devices or tracking markers that are visible in the filmbut this cannot be accomplished with existing technology.

Accordingly, there is a need for a LED panel that is utilized to createa LED volume for virtual film environments wherein the LED panel isconstructed so as to provide positional tracking of the camera whereinthe elements utilized to provide positional tracking are invisible onthe captured photographic images of the camera.

SUMMARY OF THE INVENTION

It is the object of the present invention to provide a LED panelutilized to construct LED volumes wherein the LED panel is operable toprovide positional tracking of at least one camera wherein the LED panelof the present invention is housed in a cabinet.

Another object of the present invention is to provide a LED volumeconstructed from a plurality of LED panels that provide an ability toallow filming of real time virtual environments and three dimensionaltracking of at least one camera and eliminate post-productionrequirements wherein the present invention employs embedding a pluralityof retroreflectors into the LED panels wherein the retroreflectors arenon-visible markers utilized to provide camera and/or object tracking.

A further object of the present invention is to provide a LED panelutilized to construct LED volumes wherein the LED panel is operable toprovide positional tracking of at least one camera wherein at distancesof more than a few feet the retroreflectors disappear visually to the atleast one camera and still provide infrared tracking data so as todetermine positional location of the camera.

Still another object of the present invention is to provide a LED volumeconstructed from a plurality of LED panels that provide an ability toallow filming of real time virtual environments and three dimensionaltracking of at least one camera and eliminate post-productionrequirements wherein the integrated infrared tracking retroreflector canbe embedded into the LED display panel as a small disk on the rear ofthe panel or on the front shader of the LED panel.

An additional object of the present invention is to provide a LED panelutilized to construct LED volumes wherein the LED panel is operable toprovide positional tracking of at least one camera wherein theintegrated infrared retroreflectors can further be integrated onto thepixel capsule, the enclosure that encapsulates the pixel wherein thepixel capsule can be coated with an infrared reflective chemical ormaterial that is non-visible.

Yet a further object of the present invention is to provide a LED volumeconstructed from a plurality of LED panels that provide an ability toallow filming of real time virtual environments and three dimensionaltracking of at least one camera and eliminate post-productionrequirements wherein the present invention employs an outside-in versionof camera tracking achieved by embedding a plurality of infraredretroreflectors and/or a plurality of Lidar depth sensors into the LEDpanel in alternate locations to determine the camera position in spacerelative to the infrared retroreflectors and/or Lidar depth sensors.

Another object of the present invention is to provide a LED panelutilized to construct LED volumes wherein the LED panel is operable toprovide positional tracking of at least one camera wherein the presentinvention provides discrete embedded tracking sensors in the LED panelso as to facilitate the process of capturing in-camera visual effects inan LED display in order to improve efficiency through substantialreduction of post production.

Still an additional object of the present invention is to provide a LEDvolume constructed from a plurality of LED panels that provide anability to allow filming of real time virtual environments and threedimensional tracking of at least one camera and eliminatepost-production requirements wherein the present invention provides aplurality of tracking sensors to collect depth and positional data formore accurate scanning and positional data of a camera and objects.

Yet another object of the present invention is to provide a LED panelutilized to construct LED volumes wherein the LED panel is operable toprovide positional tracking of a plurality of cameras and objectsdisposed within the LED volume constructed with the LED panels of thepresent invention.

To the accomplishment of the above and related objects the presentinvention may be embodied in the form illustrated in the accompanyingdrawings. Attention is called to the fact that the drawings areillustrative only. Variations are contemplated as being a part of thepresent invention, limited only by the scope of the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the present invention may be had byreference to the following Detailed Description and appended claims whentaken in conjunction with the accompanying Drawings wherein:

FIG. 1 is an exploded rear view of an exemplary LED panel of the presentinvention.

DETAILED DESCRIPTION

Referring now to the drawings submitted herewith, wherein variouselements depicted therein are not necessarily drawn to scale and whereinthrough the views and figures like elements are referenced withidentical reference numerals, there is illustrated a LED panel 100constructed according to the principles of the present invention.

An embodiment of the present invention is discussed herein withreference to the figures submitted herewith. Those skilled in the artwill understand that the detailed description herein with respect tothese figures is for explanatory purposes and that it is contemplatedwithin the scope of the present invention that alternative embodimentsare plausible. By way of example but not by way of limitation, thosehaving skill in the art in light of the present teachings of the presentinvention will recognize a plurality of alternate and suitableapproaches dependent upon the needs of the particular application toimplement the functionality of any given detail described herein, beyondthat of the particular implementation choices in the embodimentdescribed herein. Various modifications and embodiments are within thescope of the present invention.

It is to be further understood that the present invention is not limitedto the particular methodology, materials, uses and applicationsdescribed herein, as these may vary. Furthermore, it is also to beunderstood that the terminology used herein is used for the purpose ofdescribing particular embodiments only, and is not intended to limit thescope of the present invention. It must be noted that as used herein andin the claims, the singular forms “a”, “an” and “the” include the pluralreference unless the context clearly dictates otherwise. Thus, forexample, a reference to “an element” is a reference to one or moreelements and includes equivalents thereof known to those skilled in theart. All conjunctions used are to be understood in the most inclusivesense possible. Thus, the word “or” should be understood as having thedefinition of a logical “or” rather than that of a logical “exclusiveor” unless the context clearly necessitates otherwise. Structuresdescribed herein are to be understood also to refer to functionalequivalents of such structures. Language that may be construed toexpress approximation should be so understood unless the context clearlydictates otherwise.

References to “one embodiment”, “an embodiment”, “exemplaryembodiments”, and the like may indicate that the embodiment(s) of theinvention so described may include a particular feature, structure orcharacteristic, but not every embodiment necessarily includes theparticular feature, structure or characteristic.

Referring in particular to the Figures submitted as a part hereof, theLED panel 100 includes a housing 10 having an outer frame 12manufactured from a suitable lightweight rigid material such as but notlimited to plastic. The frame 12 includes cross members 14 wherein thecross members 14 are perpendicularly oriented between the opposing sidemembers 16, 18. It should be understood within the scope of the presentinvention that the frame 12 could have varying quantities of crossmembers depending on the size of the frame 12. While the frame 12 isillustrated as being rectangular in shape herein, it should beunderstood within the scope of the present invention that the frame 12could be provided in various shapes and sizes. Additionally, while oneLED panel 100 is illustrated herein, it should be understood within apreferred embodiment of the present invention that the LED panel 100 isutilized to construct a LED volume for virtual filming wherein the LEDvolume is constructed of numerous LED panels 100 all be of the sameconstruction as discussed herein for the LED panel 100. While notparticularly illustrated herein, it should be understood within thescope of the present invention that operation of the LED panel 100 isexecuted through conventional computing devices.

Secured to the frame 12 utilizing suitable techniques is a displayscreen 20. The display screen 20 is disposed across the entire frame andis operable to provide display of desired images to be utilized infilming virtual environments. In its preferred embodiment the displayscreen 20 is a transparent LED screen. Secured to the rear surface, orproximate thereto, of the display screen 20 are a plurality ofretroreflectors 30. The retroreflector 30 is operable to reflect lightback to its source with minimal scattering. Furthermore, theretroreflector provides a wide range of angle of incidence. Theplurality of retroreflectors are mounted to the display screen 20providing coverage of all areas thereof. The retroreflectors 30 areutilized to provide tracking of an object such as but not limited to acamera that is proximate the LED panel 100. The retroreflectors 30 arelocated so as to inhibit the retroreflectors 30 from obscuring thepixels of the display screen 20. While the retroreflectors 30 in theirpreferred embodiment are small disk shaped as illustrated herein, it iscontemplated that the retroreflectors 30 could be provided in alternateembodiments. By way of example but not limitation, the retroreflectors30 could be provided so as to be mounted in an area between the pixelsof the display screen 20 referred to as the front shader. Additionally,it is further contemplated within the scope of the present inventionthat the pixels of the display screen 20 could have a pixel capsule thatis coated with an infrared reflective chemical so as to provide thedesired function of providing light reflection for tracking of an objectproximate to the display screen 20. It should be further understoodwithin the scope of the present invention that the retroreflectors 30could be provided in all of the aforementioned embodiments within asingle LED panel 100. The retroreflectors 40 provide an ability toexecute the aforementioned methodology of inside-out tracking ofobjects. Utilization of the retroreflectors 40 and the mounting locationthereof improves the process of capturing in-camera visual effects inthe screen display 20 as the final image in camera does not need postproduction in order to remove the retroreflectors 40 from the finalimage. Furthermore, deployment of a plurality of retroreflectors 40provides an increase in available tracking data with no adverse effects.

The LED panel 100 further includes a plurality of lidar sensors 40. Thelidar sensors 40 are secured around the perimeter of the frame 12. In apreferred embodiment, the lidar sensors 40 are secured to the sidemembers 16, 18 as well as the upper cross member 7 and lower crossmember 8. The lidar sensors 40 are conventional lidar sensors whereinthe lidar sensors 40 are operable to provide light detection and rangingutilizing laser light to ping off objects so as to measure distancethereto and as such provide data for identifying the location of thepinged object. While the preferred embodiment of the LED panel 100includes a plurality of lidar sensors 40 located on the perimeter of theframe 12, it should be understood within the scope of the presentinvention that as few as one lidar sensor 40 or a plurality of lidarsensors 40 could be employed. The lidar sensors 40 are employed toprovide outside-in(as discussed previously herein) tracking of objectssuch as but not limited to a camera or prop.

The LED panel 100 further includes a plurality of optical sensors 50.The optical sensors 50 are secured to the frame 12 utilizing suitabletechniques and are located on the upper cross member 7 and lower crossmember 8. The optical sensors 50 in a preferred embodiment are opticaltriangulation position sensors operable to utilize reflected light wavesto pinpoint position of an object. The optical sensor 50 projects lightonto an object in the vicinity of the LED panel, such as but not limitedto a camera, wherein the reflected light signal is utilized to determineposition of the object. Utilization of the optical sensors 50 provides atechnique to execute the aforementioned methodology of outside-intracking of objects.

In the preceding detailed description, reference has been made to theaccompanying drawings that form a part hereof, and in which are shown byway of illustration specific embodiments in which the invention may bepracticed. These embodiments, and certain variants thereof, have beendescribed in sufficient detail to enable those skilled in the art topractice the invention. It is to be understood that other suitableembodiments may be utilized and that logical changes may be made withoutdeparting from the spirit or scope of the invention. The description mayomit certain information known to those skilled in the art. Thepreceding detailed description is, therefore, not intended to be limitedto the specific forms set forth herein, but on the contrary, it isintended to cover such alternatives, modifications, and equivalents, ascan be reasonably included within the spirit and scope of the appendedclaims.

What is claimed is:
 1. A LED panel utilized to construct a LED volumefor filming simulated virtual environments wherein the LED panelcomprises: a housing, said housing having a frame, said frame defining aperimeter edge for said housing; a display screen, said display screenhaving a front surface and a rear surface, said display screen beingmounted within said housing, said display screen operable to displayvirtual images; at least one retroreflector, said at least oneretroreflector being mounted proximate said rear surface of said displayscreen, said at least one retroreflector configured to reflect lightback to a source so as to provide positional identification of thesource; and wherein the LED panel is operable to provide a firsttechnique and a second technique of object tracking within the LEDvolume.
 2. The LED panel utilized to construct a LED volume as recitedin claim 1, and further including at least one lidar sensor, said atleast one lidar sensor being secured to said frame, said at least onelidar sensor operable to provide tracking of at least one objectutilizing the first technique.
 3. The LED panel utilized to construct aLED volume as recited in claim 2, wherein the first technique employs anoutside-in tracking methodology wherein the lidar sensors are employedto track an object disposed within the LED volume.
 4. The LED panelutilized to construct a LED volume as recited in claim 3, and furtherincluding at least one optical sensor, said at least one optical sensorbeing secured to said frame, said at least one optical sensor operableto provide the first technique of object tracking.
 5. The LED panelutilized to construct a LED volume as recited in claim 4, wherein saiddisplay screen is a transparent LED screen.
 6. The LED panel utilized toconstruct a LED volume as recited in claim 5, wherein the at least oneretroreflector is utilized to execute said second technique of objecttracking.
 7. The LED panel utilized to construct a LED volume as recitedin claim 6, wherein said at least one retroreflector is annular in shapeand proximate said rear surface of said display screen.
 8. A LED panelconfigured to be combined with additional LED panels in order to createan environment for filming simulated virtual environments wherein theLED panel comprises: a housing, said housing having a frame, said framedefining a perimeter edge for said housing, said frame having opposingside members, said frame having an upper cross member and a lower crossmember; a display screen, said display screen having a front surface anda rear surface, said display screen being mounted within said housing,said display screen being a transparent LED screen configured to displayimages; a plurality of retroreflectors, said plurality ofretroreflectors being mounted proximate said rear surface of saiddisplay screen, said plurality of retroreflectors configured to reflectlight back to a source so as to provide positional identification of thesource; a plurality of lidar sensors, said plurality of lidar sensorsconfigured to detect location of an object disposed within the LEDvolume; and wherein the LED panel is operable to provide a firsttechnique and a second technique of object tracking within the LEDvolume.
 9. The LED panel configured to be combined with additional LEDpanels in order to create an environment for filming simulated virtualenvironments as recited in claim 8, and further including a plurality ofoptical sensors, said plurality of optical sensors configured to executesaid first technique of object tracking.
 10. The LED panel configured tobe combined with additional LED panels in order to create an environmentfor filming simulated virtual environments as recited in claim 9,wherein in said first technique of object tracking signals aretransmitted from LED panel to an object disposed within the LED volume.11. The LED panel configured to be combined with additional LED panelsin order to create an environment for filming simulated virtualenvironments as recited in claim 10, wherein in said second technique ofobject tracking signals are transmitted from an object disposed withinthe LED volume towards the LED panel.
 12. The LED panel configured to becombined with additional LED panels in order to create an environmentfor filming simulated virtual environments as recited in claim 11,wherein said plurality of retroreflectors are annular in shape andmounted proximate said rear surface of the display screen.
 13. The LEDpanel configured to be combined with additional LED panels in order tocreate an environment for filming simulated virtual environments asrecited in claim 11, wherein said plurality of retroreflectors aredisposed on pixel capsules of the display screen.
 14. The LED panelconfigured to be combined with additional LED panels in order to createan environment for filming simulated virtual environments as recited inclaim 12, wherein said plurality of retroreflectors are dispersed acrosssubstantially the display screen.